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Abstract 

This paper presents a review of distance education literature published in the International Review of 
Research in Open and Distance/Distributed Learning (IRRODL) to describe the status thereof and to 
identify gaps and priority areas in distance education research based on a validated classification of 
research areas. All articles (N = 580) published between 2000 and 2015 were reviewed for this study. 
An analysis of abstracts using the text-mining tool Leximancer over three 5-year periods reveals the 
following broad themes over the three periods: the establishment of online learning and distance 
education institutions (2000-2005), widening access to education and online learning support 
(2006-2010), and the emergence of Massive Open Online Courses (MOOCs) and Open Educational 
Resources (OER) (2011-2015). The analysis auf publication and authorship patterns revealed that 
IRRODL is a very international journal with a high impact in terms of citations. 

Keywords: distance education, research, content analysis, text-mining 


Introduction 

The structure of a research discipline forms the foundation for identifying gaps and priority areas 
(Mishra, 1998). Lee, Driscoll, and Nelson (2004) emphasize that “understanding trends and issues in 
terms of topics and methods is pivotal in the advancements of research on distance education” (p. 
225). Content analysis of academic journals is a way to do this. However, highly rated peer-reviewed 
journals in distance education have been around for no more than 30-40 years. The International 
Review of Research in Open and Distance/Distributed Learning (IRRODL) and the Australian 
journal Distance Education are the only journals with a special focus on (online) distance education, 
which are listed in the Social Science Citation Index (SSCI). 

In a recent study, Zawacki-Richter and Naidu (2016) analyzed 516 articles that were published in the 
Australian journal Distance Education with the text-mining tool Leximancer 1 . The analysis revealed 


1 http://www.leximancer.com 
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the following broad research foci over seven time periods: professionalization and institutional 
consolidation (1980-1984), instructional design and educational technology (1985-1989), quality 
assurance in distance education (1990-1994), student support and early stages of online learning 
(1995-1999), the emergence of the virtual university (2000-2004), collaborative learning and online 
interaction patterns (2005-2009), and interactive learning, Massive Open Online Courses (MOOCs), 
and Open Educational Resources (OER; 2010-2014). 

This study presents a content analysis and a review of research trends and authorship patters based 
on all full articles that were published in IRRODL between 2000 and 2015 (AT = 580 articles). 
Furthermore, the aim of this analysis is to relate the results to the flow of thematic areas mentioned 
above in the journal Distance Education. 

Thus, this paper addresses questions in the following two areas: 

• Research areas covered in the publications: What are the main research topics covered in 
IRRODL publications and how have they changed between 2000 and 2015? What are the 
most common research areas and where are potential gaps in distance education research? 
What is the proportion of empirical and theoretical or descriptive papers published in 
IRRODL? Is there a trend towards more quantitative, qualitative, or mixed-method research? 

• Publication and authorship patterns: Who are the leading contributors of research papers and 
where do they come from? What are the most cited papers in IRRODL? 

Based on the review of research areas and trends, the results can be used to explore the body of 
knowledge in the field of distance/distributed learning and to identify priority areas for future 
research projects in order to support the development of a research agenda in the context of online 
distance education (Zawacki-Richter & Anderson, 2014). 


IRRODL in Previous Bibliographic Studies 

IRRODL is an open access journal published by Athabasca University Press, Canada. Teny Anderson 
was the chief editor from 2003 to 2014 and Dianne Conrad and Roiy McGreal followed him. Starting 
in 2006, IRRODL was indexed by the SSCI, which resulted in a significant increase in citations and 
greatly improved its international reputation (Martinez & Anderson, 2015). In January 2015, the 
name of the journal was changed from International Review of Research in Open and Distance 
Learning to International Review of Research in Open and Distributed Learning: “This was done to 
reflect the journal’s increased emphasis on openness in education and the blurring of boundaries in 
online learning to include blended and other forms of technology-enhanced learning” (McGreal & 
Conrad, 2016, para. 2). 

IRRODL has been the subject of previous bibliometric studies and literature reviews. Zawacki-Richter 
and Anderson (2011) carried out a citation analysis to explore the scholarly communication network of 
12 academic journals in distance education using social network analysis techniques. Although 
IRRODL is a relatively young journal, this analysis revealed that IRRODL is one of the central journals 
in the core of the journal citation network in distance education. 
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Based on the same sample of journals, Zawacki-Richter, Anderson, and Tuncay (2010) compared the 
impact of open and closed access journals including IRRODL in terms of citations and found that 
papers published in open access journals are cited earlier and more often. However, this difference 
was statistically not significant (p = 0.8). 

In another bibliometric study, Martinez and Anderson (2015) investigated the most highly cited and 
most viewed articles published in IRRODL in the time period between 2008 and 2013 (N = 401). They 
reported a high Hirsch-Index of h = 30 with an average of about eight citations per paper and 
identified a set of 33 highly cited papers, i.e., papers that were cited at least 30 times. The most cited 
paper, with 134 citations, was “Connectivism: Learning theory of the future or vestige of the past” by 
Kop and Hill (2008). Martinez and Anderson (2015) also looked at the leading authors and their 
countiy of origin and found that these highly cited articles came from 11 different countries, based on 
authors’ affiliations The most common countries identified in the highly cited articles were: United 
States (24), United Kingdom (12), and Canada (11), followed by Germany (5), Israel (5), Turkey (2), 
Norway (1), Italy (1), Denmark (1), Bahrain (1), and Australia (1). There were three contributors with 
two articles in the highly cited selection: David Wiley from the United States, Olaf Zawacki-Richter 
from Germany, and Rita Kop from Canada. 

This finding shows that IRRODL is a very international journal, which is supported by an earlier study 
of authorship patterns by Zawacki-Richter, Backer, and Vogt (2009) based on a sample of 695 articles 
published in five prominent distance education journals between 2000 and 2008. 

The most international journal is IRRODL with only 18.9% of authors from Canada, followed 
by [Distance Education] DE with 20.5% of authors from Australia, and [Open Learning] OL 
with 42.2% of authors from the UK. Papers published in IRRODL come from 34 different 
countries, followed by DE (25 countries), OL (24 countries), [the Journal of Distance 
Education ] JDE (13 countries), and [the American Journal of Distance Education ] AJDE 
(only 7 countries). (Zawacki-Richter, Backer, and Vogt, 2009, p. 40) 

Based on the same sample that included IRRODL, Zawacki-Richter and von Priimmer (2010) looked 
at gender and collaboration patterns in distance education research. Their study revealed a significant 
trend towards more collaborative research in distance education as well as significant gender 
differences noting that “Women are over-represented in research areas such as learner characteristics, 
learner support or interaction, and communication in learning communities, while men are more 
concerned with topics stereotypically associated with them: technology and management” (p. 95). 
There is also a significantly higher propensity of female researchers who apply qualitative methods 
and mixed-methods research (triangulation). 

Olsen, Spring, Young, and West (2013) examined IRRODL to determine trends in research areas and 
methods, top-authors, and top-cited publications over one decade (2002-2011). They concluded that 
“IRRODL is increasing its coverage of diverse distance education topics and is looking to publish more 
empirical data driven research than in the past....The topic analysis reflects the growing focus on 
technology-related topics in educational research” (p. 47). The authors with the highest frequency of 
contributions are Terry Anderson, Patrick Fahy, Olaf Zawacki-Richter, Rita Kop, and Mickey 
Shanchar. Interestingly, more than half of the most cited papers (8 of 14) were theoretical in nature. 
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Sample and Methods 


Papers Published in IRRODL 

For this study, all research articles published in IRRODL between 2000 and 2015 were reviewed (AT = 
580). Book reviews and editorial notes were excluded from the sample. Table 1 shows the growing 
number of papers published each year after IRRODL was included in the SSCI in 2006. Table 2 
provides an overview of all special issues (N = 17) that were published by IRRODL. 

Table 1 

Number of Articles per Year Published in IRRODL (Volumes 1-16) 


Year 

No. of 
issues 

No. of 
articles 

Year 

No. of 
issues 

No. of 
articles 

2000 

1 

6 

2008 

3 

23 

2001 

2 

13 

2009 

6 

55 

2002 

3 

3 i 

2010 

3 

20 

2003 

2 

14 

2011 

7 

51 

2004 

3 

14 

2012 

5 

59 

2005 

3 

21 

2013 

5 

71 

2006 

3 

18 

2014 

6 

86 

2007 

3 

22 

2015 

6 

76 

Total 





580 


Table 2 

Special Issues in IRRODL by Year 


Year 

Vol. 

Issue 

Topic 

2015 

16 

6 

Towards a European perspective on MOOCs 

2015 

16 

5 

OER and MOOCs 

2014 

15 

5 

Research into MOOCs 

2013 

14 

2 

OERs: Opening access to knowledge 

2012 

13 

5 

Technology-enhanced information retrieval for online learning 

2011 

12 

7 

Emergent learning, connections, design for learning 

2011 

12 

4 

Frontiers in open and distance learning in the North 

2011 

12 

3 

Connectivism: Design and delivery of social networked learning 

2011 

12 

1 

Prior, Experiential and Informal Learning in the Age of Information and 
Communication Technologies 

2009 

10 

5 

Openness and the future of higher education 

2009 

10 

4 

Open and distance learning in Africa* 

2009 

10 

2 

Learning technologies in the Middle East* 

2008 

9 

1 

Role of distance learning in the right to education 

2007 

8 

2 

Mobile Learning 

2007 

8 

1 

Open and distance education in Asia* 
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2005 6 1 Strategic enterprises down under 

2004 5 1 Low cost distance education strategies 

^Regional focus issue 

Classification of Research Areas 

The classification of research areas for this review is based on the framework developed by Zawacki- 
Richter (2009) that describes the research areas in distance education along three lines of research. 

The three levels are: 

1. Macro level: Distance education systems and theories. 

2. Meso level: Management, organization, and technology. 

3. Micro level: Teaching and learning in distance education. 

Within these three levels, the research issues were categorized into 15 research areas (for a detailed 
characterization, refer to Zawacki-Richter, 2009): 

Macro level: Distance education systems and theories. 

1. Access, equity, and ethics. 

2. Globalization of education and cross-cultural aspects. 

3. Distance teaching systems and institutions. 

4. Theories and models. 

5. Research methods in distance education and knowledge transfer. 

Meso level: Management, organization, and technology. 

6. Management and organization. 

7. Costs and benefits. 

8. Educational technology. 

9. Innovation and change. 

10. Professional development and faculty support. 

11. Learner support services. 

12. Quality assurance. 
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Micro level: Teaching ancl learning in distance education. 

13. Instructional design. 

14. Interaction and communication in learning communities. 

15. Learner characteristics. 

All 580 articles published in IRRODL between 2000 and 2015 were coded according to this 
classification scheme. 

Computer-Assisted Content Analysis 

For the purposes of this study, the content analysis software Leximancer (2011 version) was used to 
produce a set of concept maps showing the semantic structure of themes and topics of articles and 
how they are related. Fisk, Cherney, Hornsey, and Smith (2012) indicate that computer-aided content 
analysis is an appropriate method to map a research domain. The software Leximancer has been used 
by other researchers for content analysis of academic journals, e.g., the Journal of Cross-Cultural 
Psychology (Cretchley, Rooney, & Gallois, 2010), the Journal of International Business Studies 
(Liesch, Hakanson, McGaughey, Middleton & Cretchley, 2011), the Journal of Communication (Lin & 
Lee, 2012), and Distance Education (Zawacki-Richter & Naidu, 2016). 

The software identifies core concepts within textual data (conceptual analysis) and how these concepts 
are related (relational analysis) by the frequency with which words co-occur in the text. Similar 
concepts that co-occur in close proximity are clustered in the visual map produced by Leximancer: 
“The map is an indicative visualization that presents concept frequency (brightness), total concept 
connectedness (hierarchical order of appearance), direct interconcept relative co-occurrence 
frequency (ray intensity), and total (direct and indirect) interconcept co-occurrence (proximity)” 
(Smith & Humphreys, 2006, p. 264). Depending on the connectedness of concepts they form a 
thematic region. Such a “theme” is named after the most prominent concept in that group. 

Titles and abstracts are lexically dense and focus on the core themes and results presented in the 
articles. An initial overall analysis was run with abstracts and titles of 580 articles, in which common 
terms (and, not, etc.) were excluded. In addition, Leximancer was requested to merge word variants, 
e.g., distance and education or open and university. 

For the purposes of this study, Leximancer was used to analyze both the entire data set (2000-2015) 
and each 5-year time period separately. Abstracts and titles of all articles published in IRRODL 
between Volume 1 (1) in 2000 and Volume 16 (6) in 2015 were collected. A total of 580 full papers, 
and subsets of data in three 5-year periods were created: 2000-2005 2 (99 articles), 2006-2010 (138 
articles), and 2011-2015 (343 articles). 


2 The first issue of IRRODL was published at the end of 2000. This issue was added to the first 5-year time period from 2001- 
2005. 
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Limitations 

We acknowledge that IRRODL is only one window to look into the structure and flow of research 
areas in online distance and distributed learning and the selection of the sample for the purposes of 
this study is subject to interferences: 

The most important of these is surely the gatekeeping role of reviewers, editors and journals 
in general. Quite aside from what one might prefer to do, publication responds to funding 
possibilities and publishing possibilities, and these in turn respond to connections and 
selection of a topic, a method, and a choice of potential journal most likely to lead to 
publication. (Goldenberg & Grigel, 1991, p. 436) 

A text-mining tool such as Leximancer is a powerful instrument for the purpose of mapping a research 
domain (see Fisk, Cherney, Hornsey, & Smith, 2012), but we have to acknowledge that the results 
must be interpreted with profound knowledge of the subject matter. Leximancer is known to produce 
stable results as has been noted by Harwood, Gapp, and Stewart (2015). Their analysis revealed 
encouraging similarities between a Leximancer output and main themes and codes derived from a 
manual Grounded Theory analysis. However, they remind us that 

Leximancer is not a panacea, it still requires analytical sensitivity and judgment in its 
interpretation, but it is straightforward to probe the data and cross-check via the resultant 
maps....Leximancer enables the analyst to make sense of large narrative data sets with 
minimal manual coding. The result is an efficient and impartial second opinion on open codes 
(concepts, categories and dimensions) and potential links between them. (Harwood. Gapp, & 
Stewart, 2015, p. 1041) 


Results and Discussion 

Content Analysis: Research Areas and Trends 

Overall scope of the journal (2000-2015). The concept map in Figure 1 depicts the major 
topics covered in the articles published over the first 16 years of the journal (2000-2015). The 
thematic summary includes a connectivity score to indicate the relative importance of the themes. The 
results reveal that learning has the most direct mentions within the text with 276 (100% relative 
count), followed by study (66%), online (45%), distance (42%), social (13%), and resources (7%). This 
is not a surprising result, as this journal deals with the study of online distance education. While this 
may seem like a trivial observation, it confirms the veracity of Leximancer as a useful text-mining tool. 


7 



Review and Content Analysis of the International Review of Research in Open and Distance/Distributed Learning (2000-2015) 

Zawacki-Richter, Alturki, and Aldraiweesh 




theory 


courses 


traditional 


e-learning 


knowledge 


learning 


framework 



interaction 


research 


social 



teachers 




system 

oM ine 





learners 


social 

support 

community 

content 

study 

study 




students 


communication 




analysis 


university 


development 

learning 


academic 

teaching 


r °l e distance 

distance 


Figure l. Concept map of overall scope of the journal IRRODL (2000-2015). 

A key focus of the articles published in IRRODL as an open access journal is the potential and role of 
open and distance learning to provide access to education and educational opportunities. The 
concepts role, distance, and education are directly connected with educational, resources, and open 
in the concept map (see Figure 1). 

Two major topics covered in the publications (i.e., learning and online ) are connected via the thematic 
region social, including the concepts of interaction, communication, and support. Thus, learning is 
seen as a social process that is facilitated by interaction among learners and teachers. The provision of 
opportunities for interaction, communication, and collaboration between students and their teachers 
as well as among students, via electronic media, is seen as a constituent element of distance education 
(Keegan, 1980). 
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Table 3 

Distribution of Research Areas From 2000-2015 (N = 580) 


Rank 

F 

Research area 

1 

103 

Instructional design 

2 

82 

Learner characteristics 

3 

74 

Educational technology 

4 

69 

Interaction and communication in learning communities 

5 

42 

Distance teaching systems and institutions 

6 

41 

Professional development and faculty support 

7 

34 

Theories and models 

8 

29 

Access, equity, and ethics 

9 

27 

Quality assurance 

10 

19 

Research methods in distance education and knowledge transfer 

11 

16 

Management and organization 

12 

12 

Costs and benefits 

12 

12 

Learner support services 

13 

10 

Globalization of education and cross-cultural aspects 

13 

10 

Innovation and change 

Total 

580 



In the three 5-year intervals there are recurring concepts, such as students, online, and education, that 
refer to very broad and common thematic areas throughout the whole time period. In the following 
content analysis and interpretation, emphasis is placed on new and emerging concepts in order to 
describe the changes and trends in distance education research. 

Table 3 provides an overview of the 15 research areas in terms of their frequency covered in the 
publications. The top issues are related to the micro-level of teaching and learning in online distance 
education: instructional design, learner characteristics, interaction and communication in learning 
communities, and the application of educational technology to design flexible learning opportunities. 
Research areas on the bottom of the list are costs and benefits and learner support services (with 12 
papers) and globalization of education and innovation and change (with only 10 papers). This 
corresponds with earlier findings of Zawacki-Richter, Backer, and Vogt (2009). 

Online learning and distance education institutions (2000-2005). The major 
themes emerging between 2000 and 2005 are education (100%), students (60%), online (24%), 
development (14%), research (7%), and analysis (4%) (see Figure 2). The mid-1990s saw growing 
interest in online learning and the transition to online teaching institutions, which was an enormous 
process of innovation. Zawacki-Richter and Naidu (2016) call the time period between 2000 and 2004 
“the emergence of the virtual university” (p. 257). 

Accordingly, many papers deal with the challenges, opportunities, and processes of introducing online 
learning at higher education institutions. The concepts education, higher, processes, institutions, and 
delivery are directly connected in the concept map. Also, the results of the manual coding of research 
areas that are reported in Table 4 show that “Distance teaching systems and institutions” is the top 
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research issue, followed by “Interaction and communication in learning communities,” and “Theories 
and models.” There are papers that focus on the development of online course delivery and online 
student support services worldwide, such as Deakin University as an example for a dual-mode 
university in Australia (Calvert, 2001), the NKI Internet College in Norway (Paulsen & Rekkedal, 

2001) , or the Indira Ghandi National Open University (IGNOU) in India (Sharma, 2001). Other 
articles consider the “hybridization” of whole national higher education systems, such as Brazil (Litto, 

2002) , or Canada (Shale, 2002). 
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Figure 2. Concept map of online learning and distance education institutions (2000-2005). 

Theoretical papers in this time period elaborate on the pedagogical opportunities that online learning 
affords. Peters (2000) explores pedagogical and didactical activities in online learning environments 
and develops a theory for digital learning spaces, and Anderson (2003) discusses the role of new 
information and communication technologies to optimize the mix between independent study and 
interactive learning strategies and activities. 
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Emphasis in empirical research is placed on the students in this new online learning environment. 
Especially, the building and sustaining a sense of community in the online classroom is a major focus 
(Rovai, 2002). Researchers investigate questions such as: 

Does a social atmosphere develop in online learning discussion groups? What are the different 
modes of social interaction are manifest in online learning discussion groups? What is the role 
of the virtual teacher with regards to the social climate in online learning discussion groups? 
(Oren, Mioduser, & Nachmia, 2002, p. 1) 

It is interesting to look at the research areas that are at the bottom of the ranking in terms of the 
number of published articles in this time period (Table 4): costs and benefits, professional 
development and faculty support, and access, equity, and ethics. Researchers turned their attention to 
these neglected research areas in the next time period. 

Table 4 

Distribution of Research Areas From 2000-2005 (N = 99) 


Rank 

F 

Research area 

1 

26 

Distance teaching systems and institutions 

2 

14 

Interaction and communication in learning communities 

3 

10 

Theories and models 

4 

7 

Educational technology 

4 

7 

Learner support services 

4 

7 

Quality assurance 

4 

7 

Instructional design 

5 

4 

Globalization of education and cross-cultural aspects 

5 

4 

Learner characteristics 

6 

3 

Management and organization 

6 

3 

Innovation and change 

7 

2 

Research methods in distance education and knowledge transfer 

7 

2 

Costs and benefits 

7 

2 

Professional development and faculty support 

8 

1 

Access, equity, and ethics 

Total 

99 



Widening access to education and online learning support (2006-2010). In this 
second half of the first decade in the new millennium, conventional distance education is in the 
transition to online learning. As Figure 3 shows, the major topics canvassed in publications over these 
five years are: students (100%), education (54%), university (15%), open (9%), support (6%), and 
social (2%). Online and learning are central concepts in the thematic region of students. 
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Figure 3. Concept map of widening access to education and online learning support (2006-2010). 

Around this time the issue of access to educational opportunities and open education came to the fore, 
moving up from the eighth to the third rank of research areas covered in the publications (see research 
area of “access, equity, and ethics” in Table 5). Four special issues deal with all kinds of aspects related 
to openness and open education: Open and Distance Education in Asia (2007), The Role of Distance 
Learning in the Right to Education (2008), Open and Distance Learning in Africa (2009), and 
Openness and the Future of Higher Education (2009). Papers in this thematic context explore the 
emergence of open-source software, course management systems, in particular, in North America 
(Pan & Bonk, 2007), the role of using ICTs and open and distance learning in increasing access, 
equity, and quality of rural teachers professional development in China or the potential of open access 
to reduce the high costs of textbooks at American universities (Baker, Thierstein, Fletcher, Kaur, & 
Emmons, 2009). Fini (2009) published the first paper in IRRODL about the evaluation of a MOOC, 
the famous Connectivism and Connective Knowledge (CCK08) course. 

There is also growing awareness among scholars and practitioners at this time of the importance of 
professional development and faculty support, which is critical for the development of high-quality 
online programs and education innovation (Brindley, Zawacki, & Roberts, 2003). The concept map 
shows that online, learning, and development, and faculty are linked via the concept quality (see 
Figure 5). For example, Villar and Alegre (2006) present an online faculty development and 
assessment system: “The rapid growth of online learning has led to the development of faculty 
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evaluation models that are geared towards the demands of quality improvement of degree programs” 
(p. l). Also, Chao, Saj, and Hamilton (2010) highlight that “the issue of quality is becoming front and 
center as online distance education moves into the mainstream of higher education” (p. 106). Eib and 
Miller (2006) see faculty development as an on-going need and long-term, continuous effort and 
emphasize the importance of community building “to create a culture that supports a thoughtful focus 
on teaching” (p. 1). 

Table 5 

Distribution of Research Areas from 2006-2010 (N = 138) 


Rank 

F 

Research area 

1 

28 

Instructional design 

2 

21 

Learner characteristics 

3 

17 

Access, equity, and ethics 

4 

14 

Educational technology 

4 

14 

Professional development and faculty support 

5 

9 

Interaction and communication in learning communities 

6 

8 

Distance teaching systems and institutions 

6 

8 

Research methods in distance education and knowledge transfer 

7 

6 

Quality assurance 

8 

4 

Management and organization 

8 

4 

Innovation and change 

9 

3 

Theories and models 

10 

1 

Globalization of education and cross-cultural aspects 

10 

1 

Costs and benefits 

11 

0 

Learner support services 

Total 

138 



Although support emerged as a concept that forms a thematic region of its own, and 14 papers were 
published that deal with professional development and faculty support, there are interestingly no 
articles between this 5-year period that deal with student support service systems (see Table 5) on the 
institutional level of educational management (meso-level). In contrast, Zawacki-Richter and Naidu 
(2016) found that the provision of student support services was a major research area in the 1990s, 
when student support was acknowledged as a “critical link in distance education” (Dillon, 
Gunawardena, & Parker, 1992, p. 29). 

However, researchers around this time are fascinated by the enormous opportunities that the new 
information and communication technologies afford for collaborative online learning and teaching 
(micro-level). There are several publications in IRRODL that seek to understand how online 
collaborative learning occurs and how students are best supported to facilitate their learning 
processes: Barnard-Brak, Paton, and Lan (2010) developed a typology of self-regulated learning 
profiles in the online learning environment and several studies deal with the role and function of 
online tutors (e.g., Ng, 2007; Murugaiah & Thang, 2010). The concept of support is connected with 
access. For example, Long, Vignare, Lappold, and Mallory (2007) reported on a study that 
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investigated how to provide access to computer-mediated communication for learners with special 
needs like deaf, hard-of-hearing, and ESL students. 

The emergence of MOOCs and OER (2011-2015). Over the next five years (2011- 
2015), the themes students (100%) and education (56%) continue to be critical followed by teaching 
(36%), courses (20%), system (18%), MOOCs (5%), and OER (3%). From 2011-2015 four special 
issues were released about open education, especially MOOCs and OER. These issues were titled: 
Towards a European perspective on MOOCs (2015), OER and MOOCs (2015), Research into MOOCs 
(2014), and OERs: Opening Access to Knowledge (2013). Another special issue in 2011 dealt with 
Prior, Experiential and Informal Learning in the Age of Information and Communication 
Technologies (PLAR), which is also closely related to openness in (higher) education (see Conrad, 
2010). 

The concept of students, online, courses, and MOOCs are directly connected in the concept map (see 
Figure 6). Many papers describe and evaluate the development of MOOCs for different target groups: 
from the integration of MOOCs into secondary school courses (Najafi, Evans, & Federico, 2014) or the 
area of corporate training (Radford et al., 2014) to contexts of limited resources such as developing 
countries in Africa (Oyo & Kalema, 2014). Open education does not mean that education is free, as 
someone has to pay the bill for the sustainable development and delivery of courses and course 
materials. The issue of costs and economics of open and distance/distributed learning moves up in the 
ranking slightly in terms of frequency covered in the publications (see Table 6). Some authors raise 
the question of costs for the development and delivery MOOCs (Hollands & Tirthali, 2014) and 
financial sustainability of open textbooks (Iii & Wiley, 2011). 
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Figure 4. Concept map of MOOCs and OER (2011-2015). 

Proponents of OER claim that significant cost savings are possible when open textbooks replace 
traditional textbooks in the classroom. However, Wiley, Iii, Ellington, and Hall (2012) report 
contradicting findings as open textbooks are deployed in various ways: “Some of these methods cost 
more than traditional textbooks; however, we did identify and implement a successful model of open 
textbook adoption that reduces costs by over 50% compared to the cost of adopting traditional 
textbooks” (p. 262). 

Professional development and faculty support remains an important topic in the thematic region of 
teaching (see concept path teachers - professional - development). This research area is also ranked 
fifth in Table 6. Papers published in IRRODL notably explored the potential of open and distance 
learning for teacher training and the development of competencies for the implementation and use of 
ICTs in the teaching and learning process (e.g., van Zyl, Els, & Blignaut, 2013; Mhishi, Bhukuvhani, & 
Sana, 2012). 
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Table 6 

Distribution of Research Areas from 2011-2015 (N = 343) 


Rank 

F 

Research area 

1 

68 

Instructional design 

2 

57 

Learner characteristics 

3 

53 

Educational technology 

4 

46 

Interaction and communication in learning communities 

5 

25 

Professional development and faculty support 

6 

21 

Theories and models 

7 

14 

Quality assurance 

8 

11 

Access, equity, and ethics 

9 

9 

Research methods in distance education and knowledge transfer 

9 

9 

Management and organization 

9 

9 

Costs and benefits 

10 

8 

Distance teaching systems and institutions 

11 

5 

Globalization of education and cross-cultural aspects 

11 

5 

Learner support services 

12 

3 

Innovation and change 

Total 

343 



Methods, Publication, and Authorship Patterns 

Leading authors. The total number of different authors who contributed to the 580 articles 
in this study was 1157. Table 7 lists the 40 authors who contributed to at least three articles. They are 
from the USA (12), Canada (11), South Africa (4), Australia (2), Israel (2), Denmark (1), Germany (1), 
the Netherlands (1), Norway (1), Portugal (1), Spain (1), Turkey (1), and the UK (1). The top author 
with nine publications in IRRODL is Teny Anderson from Athabasca University in Canada. He was 
the chief editor of IRRODL from 2003 to 2014. Zawacki-Richter, Anderson, and Tuncay (2010) 
showed that there is a positive bias of editors towards their own journal. 

Table 7 

Leading Contributors and Number of Their Published Articles 


Anderson, Terry (Canada) 9 

Wiley, David A. (USA) 7 

Hilton, John Levi (USA) 6 

Zawacki-Richter, Olaf (Germany) 6 

Jung, Insung (South Korea) 5 

Fahy, Patrick J. (Canada) 5 

Bullen, Mark (Canada) 4 

Chetty, Yuraisha B. (S. Africa) 4 

Graham, Charles R. (USA) 4 

Mackness, Jenny (UK) 4 

McGreal, Roiy (Canada) 4 

Murphy, Elizabeth (Canada) 4 


Blau, Ina (Israel) 3 

Brown, Abbie H. (USA) 3 

Chen, Yong (USA) 3 

Ching, Yu-Hui (USA) 3 

Conrad, Dianne (Canada) 3 

Dalsgaard, Christian (Denmark) 3 

Gokool-Ramdoo, Sushita (Australia) 3 

Gorsky, Paul (Israel) 3 

Howell, Scott L. (USA) 3 

Kenny, Richard F. (Canada) 3 

Kop, Rita (Canada) 3 

Mbati, Lydia S. (South Africa) 3 
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Prinsloo, Paul (South Africa) 4 

Schuwer, Robert (Netherlands) 4 

Veletsianos, George (USA) 4 

Ally, Mohamad (Canada) 3 

Annand, David (Canada) 3 

Archer, Elizabeth (South Africa) 3 

Aydin, Cengiz Hakan (Turkey) 3 

Barbour, Michael K. (USA) 3 


Najafi, Hedieh (Canada) 3 

Nunan, Ted (Australia) 3 

Paulsen, Morten Flate (Norway) 3 

Sangra, Albert (Spain) 3 

Shea, Peter (USA) 3 

Teixeira, Antonio (Portugal) 3 

Walker, Andrew E. (USA) 3 

West, Richard E. (USA) 3 


Gender and research methods. The analysis in Table 8 depicts that 324 (55.9%) of the 
first authors were men and 256 (44.1%) were women. The results appear to confirm the stereotypical 
view that female researchers (64.2%) are more likely than males (35.8%) to choose qualitative 
methods. The same pattern was found by Zawacki-Richter and von Priimmer (2010) looking at papers 
published in five distance education journals between 2000 and 2008. There is a highly significant 
association between gender and research methods, x 2 = 27.62, df = 3, p < .001. However, the 
association is modest at Cramer's V of .22 (p < .001). 

Overall, there seems to be a tendency towards more empirical research. Only 44.5% of all articles 
published IRRODL are theoretical or descriptive in nature, whereas Zawacki-Richter, Backer, and 
Vogt (2000) reported that IRRODL was the journal that accepted the highest number of descriptive or 
theoretical papers (56.6 %) between 2000 and 2008. 

Table 8 

Cross Tabulation of Gender (First Author) and Research Methods 





Male 

Female 

Total 

Method 

Quantitative 

Count 

% within method 

92 

57-9% 

67 

42.1% 

159 

100.0% 


Qualitative 

Count 

% within method 

39 

35-8% 

70 

64.2% 

109 

100.0% 


Triangulation 

Count 

% within method 

26 

48.1% 

28 

51.9% 

54 

100.0% 


Other 

Count 

% within method 

167 

64.7% 

9i 

35-3% 

258 

100.0% 

Total 


Count 

% of total 

324 

55-9% 

256 

44.1% 

580 

100.0% 


Countries. For the analysis of the geographic distribution of articles, the countiy of origin of 
the first author was taken into consideration (64 countries, see Figure 5). The results confirm the 
earlier finding by Zawacki-Richter, Backer, and Vogt (2009) that IRRODL is a very international 
journal. The articles came from 64 different countries (see Table 9), the majority from the USA, 
Canada, the UK, Australia, South Africa, Spain, Germany, and Turkey. 
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Table 9 

Distribution of Author’s Country of Origin for Articles in IRRODL 


Country* 

Total 

% 

Cum. % 

Country 

Total 

% 

Cum. % 

USA 

153 

VO 

CM 

26.4 

South Korea 

3 

•5 

92.4 

Canada 

89 

15-3 

41.7 

Pakistan 

3 

•5 

92.9 

UK 

4i 

7-t 

48.8 

Philippines 

3 

•5 

93-4 

Australia 

28 

4-8 

53-6 

Thailand 

3 

•5 

94-0 

South Africa 

22 

3-8 

57-4 

Estonia 

2 

•3 

94-3 

Spain 

22 

3-8 

61.2 

Indonesia 

2 

•3 

94-7 

Germany 

18 

3-t 

64-3 

Ireland 

2 

•3 

95-0 

Turkey 

18 

3-t 

67.4 

Mexico 

2 

•3 

95-3 

Israel 

14 

2.4 

69.8 

Russian 

Federation 

2 

•3 

95-7 

Taiwan 

14 

2.4 

72.2 

Zambia 

2 

•3 

96.0 

The Netherlands 

10 

1-7 

74-0 

Zimbabwe 

2 

•3 

96.4 

Sweden 

9 

1.6 

75-5 

Armenia 

1 

.2 

96.6 

China 

8 

1-4 

76.9 

Bahrain 

1 

.2 

96.7 

New Zealand 

8 

1-4 

78.3 

Bangladesh 

1 

.2 

96.9 

India 

7 

1.2 

79-5 

Barbados 

1 

.2 

97.1 

Norway 

7 

1.2 

80.7 

Cyprus 

1 

.2 

97.2 

South Korea 

7 

1.2 

81.9 

Hong Kong 

1 

.2 

97-4 

Nigeria 

6 

1.0 

82.9 

Iceland 

1 

.2 

97-6 

Botswana 

5 

•9 

83.8 

Italy 

1 

.2 

97-8 

Japan 

5 

•9 

84.7 

Lithuania 

1 

.2 

97-9 

Malaysia 

5 

•9 

85-5 

Macedonia 

1 

.2 

98.1 

Brazil 

4 

•7 

86.2 

Mauritius 

1 

.2 

98.3 

Greece 

4 

•7 

86.9 

Oman 

1 

.2 

98.4 

Portugal 

4 

•7 

87.6 

Qatar 

1 

.2 

98.6 

Serbia 

4 

•7 

88.3 

Rwanda 

1 

.2 

98.8 

Colombia 

3 

•5 

88.8 

Saudi Arabia 

1 

.2 

99.0 

Denmark 

3 

•5 

89-3 

Slovakia 

1 

.2 

99.1 

Finland 

3 

•5 

89.8 

Switzerland 

1 

.2 

99-3 

France 

3 

•5 

90.3 

Trinidad and 
Tobago 

1 

.2 

99-5 

Ghana 

3 

•5 

90.9 

Ukraine 

1 

.2 

99-7 

Iran 

3 

•5 

91.4 

Venezuela 

1 

.2 

99.8 

Kenya 

3 

•5 

91.9 

Vietnam 

1 

.2 

100.0 


“only the country of the first author was taken into consideration here. 
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Figure 5. Countries of origin of first authors. 

Top-cited articles and authors. Some journals put a strong emphasis on empirical work 
and discourage, or even disallow, theoretical work, including literature reviews. However, based on a 
series of literature reviews of educational technology journals, West (2016) consistently found that 

theoretical/literature-based articles to be among the most cited in a journal. For example, in 
Educational Technology Research and Development, when looking at the top-cited articles 
from each year in the decade, seven were theoretical articles, not reporting new empirical 
findings, (p. 44) 

The following table includes the top 10 most cited articles for the entire time period, as well as the 
most cited article from each year. The results of the citation analysis confirm the findings by West 
(2016). Only eight out of the 18 most cited articles are empirical in nature, using quantitative, 
qualitative, or mixed-methods analysis. Theoretical work is important, and obviously, readers and 
scholars value these articles. 

Table 10 

Most Cited Papers From Years 2000-2015 for IRRODL 


Citations Author(s) Title Year Method 


7 

A. Bozkurt et 
al. 

Trends in distance education research: a content 
analysis of journals 2009-2013 

2015 

content 

analysis 

136 

K. Jordan 

Initial trends in enrolment and completion of 
Massive Open Online Courses 

2014 

quantitative 

283 

T. R. 

Liyanaguna- 
wardena et 
al. 

MOOCs: A systematic study of the published 
literature 2008-2012 

2013 

content 

analysis 
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123 

B. Chen, 

T. Bryer 

Investigating instructional strategies for using 
social media in formal and informal learning 

2012 

qualitative 

418 

T. Anderson, 
J. Dron 

Three generations of distance education 
pedagogy 

2011 

theoretical 

322 

R. Kop 

The challenges to connectivist learning on open 
online networks: learning experiences during a 
massive open online course 

2011 

mixed 

315 

Y. Park 

A pedagogical framework for mobile learning: 
categorizing educational applications of mobile 
technologies into four types 

2011 

theoretical 

157 

J.H. Valk et 
al. 

Using mobile phones to improve educational 
outcomes: an analysis of evidence from Asia 

2010 

quantitative 

238 

A. Fini 

The technological dimension of a massive open 
online course: the case of the CCK08 course 
tools 

2009 

quantitative 

374 

R. Kop, A. 

Hill 

Connectivism: Learning theory of the future or 
vestige of the past? 

2008 

theoretical 

546 

J. Traxler 

Defining, discussing, and evaluating mobile 
learning: the moving finger writes and having 
writ.... 

2007 

theoretical 

200 

E. J. Stodel et 
al. 

Learners' Perspectives on What is Missing from 
Online Learning: Interpretations through the 
Community of Inquiry Framework 

2006 

qualitative 

122 

P. Shea et al. 

Increasing access to Higher Education: A study 
of the diffusion of online teaching among 913 
college faculty 

2005 

quantitative 

594 

A. P. Rovai, 

H. M.Jordan 

Blended learning and sense of community: a 
comparative analysis with traditional and fully 
online graduate courses 

2004 

quantitative 

590 

T. Anderson 

Getting the mix right again: an updated and 
theoretical rationale for interaction 

2003 

theoretical 

986 

A. P. Rovai 

Building sense of community at a distance 

2002 

theoretical 

368 

S. Downes 

Learning objects: resources for distance 
education worldwide 

2001 

theoretical 

545 

R. Garrison 

Theoretical challenges for distance education in 
the 21st Century: A Shift from Structural to 
Transactional Issues 

2000 

theoretical 
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Conclusion 

The aim of this study was to investigate the research areas and trends that are covered in publications 
in IRRODL and the authorship patterns that apply to these areas. In conclusion, the research areas 
covered in the three 5-year time periods can be described as follows: 1) Online learning and distance 
education institutions (2000-2005), 2) Widening access to education and online learning support 
(2006-2010), and 3) The emergence of MOOCs and OER (2011-2015). The mapping of the contents 
of IRRODL can be related to the flow of thematic areas over roughly the time periods in the journal 
Distance Education (see Zawacki-Richter & Naidu, 2016): The emergence of the virtual university 
(2000-2004), Collaborative learning and online interaction patterns (2005-2009), and interactive 
learning, MOOCs, and OER (2010-2014). This indicates high similarity between major research 
topics covered in the publications in these two journals. However, IRRODL has a stronger focus on 
open education, which is also reflected in the name change from the International Review of Open and 
Distance Learning to the International Journal of Open and Distributed Learning (McGreal & 
Conrad, 2016). 

With regard to the publication and authorship patterns, major findings of this study may be 
summarized as follows: 

• The 580 contributions published in IRRODL originated from 64 countries, but the majority 
from wealthy western nations like the USA, Canada, the UK, Australia, South Africa, Spain, 
and Germany. However, the results also show that IRRODL is an important outlet for scholars 
from many different and developing countries as well. 

• The citation analysis revealed a range of highly cited articles that were published in IRRODL, 
and interestingly the top-cited papers are theoretical in nature. This finding supports the fact 
that theoretical papers are important and valued not only by IRRODL's readership but also 
recognized as influential on the field at large. 

• Nevertheless, there seems to be a trend towards more studies in IRRODL that apply empirical 
research methods (qualitative, quantitative, and mixed). Over the whole time period, only 
44.5% of the research articles are theoretical or descriptive in nature. 

All in all, the name tells the story: The International Review of Research in Open and Distributed 
Learning is a very international journal with articles from over 60 countries and with a major focus in 
the publications placed on openness and access to educational opportunities. IRRODL is an excellent 
outlet for open scholars worldwide to communicate and share their research findings in a high impact 
journal. 
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